Delicate Interpretation , Illusion and Feedforward : Three Key Concepts toward Designing Multimodal Interaction

This paper argues that delicate interpretation, illusion, and feedforward as three key concepts in designing future multimodal interaction. The human brain has its peculiar nature of integrating information coming through different sensory channels to construct a consistent model of the world. The notion of direct manipulation and feedback based on the truthful reflection of the physical world may no longer be the guiding framework for designing tangible, embedded, and embodied interaction. HCI designers (as well as brain scientists) have very limited understanding on how the brain models the external world by using multimodal information. TCieX (Touch-Centric interaction embodiment eXploratorium) has been built to help us experience and understand the combinations of different modes of interaction, and explore the three key concepts in designing multimodal interaction.